Head-mounted display devices may receive user inputs from a pointing device, such as a mouse or touch-sensitive surface. When displaying a virtual representation of a desktop to a user, such inputs may control a pointer that may interact with elements of the desktop. However, movement of the pointer is confined to the virtual desktop representation and the user is limited to interacting with elements of the desktop. Further, desktop elements such as applications that are displayed on the desktop are also confined to the area of the desktop representation.